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INTRODUCTION 

HiS-tary  and  Purpose 

Recognizing  the  benefits  of  an  ambulatory  care  data  base,  the  Army 
Medical  Department  began  planning  in  1984  for  a  multi-year  study  to  establish 
an  outpatient  data  base.  Based  on  the  results  of  a  6-month  pilot  study 
r  completed  at  Fox  Army  Community  Hospital,  Redstone  Arsenal,  Alabama  (Misener  & 

Gilbert,  1984),  the  ACOB  Study  (Georgoulakis  et  al,  1988)  was  initiated  to 
collect  clinical  data  from  outpatient  encounters  (visits).  During  the  21- 
month  period  of  the  study  (January  1986  to  September  1987),  over  3.1  million 
patients  encounters  were  recorded  from  the  six  study  sites,  representing  more 
than  4,000  health  care  providers  in  some  70  clinical  specialties. 

This  report  examines  the  more  salient  aspects  of  the  study  from  the 
participating  health  care  providers'  perspective.  In  order  to  quantify 
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provider  input,  a  structured  questionnaire  was  employed. 


Backguftund 

At  the  present  time  the  practice  of  medicine  in  both  the  military  and 
civilian  communities  is  experiencing  a  great  deal  of  change.  Researchers  are 
constantly  conducting  studies  looking  for  ways  to  increase  the  availability 
of  medical  care  and  at  the  same  time  reduce  the  cost  of  the  care. 
Unfortunately,  during  the  course  of  these  studies  reseachers  frequently  fail 
to  provide  study  participants  with  an  opportunity  to  furnish  input  or  to 
evaluate  the  project.  The  ACDB  study  group  attempted  to  overcome  this  failure 
by  incorporating  both  formal  and  informal  program  evaluation  methods  into  the 
study  design.  The  informal  methods  were  designed  to  measure  the  impact  of  the 
study  from  the  provider  perspective. 

OBJECTIVES 

The  major  objectives  of  this  study  were  to 

1.  Solicit  input  from  the  provider  participants  on  the  adequacy  of  the 
individual  specialty  menus  on  the  patient  care  collection  instruments. 

2.  Provide  a  self-report  measure  from  the  provider's  perspective  on  the 
accuracy  of  the  data  contained  in  the  clinical  sections  of  the  "bubble"  forms. 

3.  Obtain  a  measure  of  the  amount  of  time  required  by  the  providers  to 
complete  the  patient  encounter  (care)  forms  (commonly  called  buoble  forms). 

4.  Provide  the  study  providers  with  a  forum  to  express  their  views  on 
the  project. 


METHODOLOGY 

Development  flf  Provider  Sur.YSy 

Based  on  the  objectives  of  the  study,  the  ACDB  study  team  initiated  a 
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series  of  work  sessions  to  determine  the  best  method  to  reach  the  stated 
objectives.  After  reviewing  a  number  of  options,  the  study  team  concluded 
that  a  self-administered  questionnaire  would  be  most  appropriate.  A  survey 
was  decided  on  by  the  study  team  because  it  would  allow  mailings  to  the  study 
sites  and,  more  importantly,  would  allow  the  providers  the  flexibility  of 
completing  the  surveys  when  their  busy  schedules  permitted.  Additionally,  the 
use  of  a  survey  was  the  most  advantageous  in  terms  of  economic  and  time 
constraints. 

Contents  flf  tii£  Provider  Survey 

To  insure  that  the  providers  would  be  able  to  coiranent  on  as  many  aspects 
of  the  ACDB  project  as  possible,  the  study  team  began  by  developing  an  item 
pool  of  questions  centering  around  five  major  areas:  (1)  demographics,  (2) 
utilization  of  the  old  patient  encounter/bubble  forms  (forms  used  prior  to  1 
May  1987  were  designated  as  old  forms),  (3)  utilization  of  the  new  forms 
(forms  used  after  1  May  1987  were  designated  as  the  new  forms) ,  (4)  a 
comparison  of  the  old  and  new  forms  with  comments  and  (5)  a  section  for 
clinical  chiefs. 

The  demographic  section  consisted  of  three  questions  and  was  designed  to 
provide  information  on  the  rank  and  pay  grade  of  the  provider,  the  length  of 
time  the  individual  had  been  working  in  a  medical  treatment  facility,  and  the 
specialty  area  in  which  they  worked  most  of  the  time. 

The  second  section  contained  ten  questions  relating  to  the  old  bubble 
form.  Questions  were  developed  around  frequency  of  use,  time  required  to 
complete  the  form,  accuracy  of  information  entered  in  the  forms,  and  adequacy 
of  clinical  menus. 

The  third  section  consisted  of  eleven  questions  relating  to  the  new 
bubble  forms.  With  the  exception  of  the  one  additional  question  which 
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related  to  workload,  the  questions  in  section  two  and  three  were  identical. 

Section  four  was  designed  for  users  of  both  the  old  and  new  forms.  This 
section  contained  two  questions  pertaining  to  a  comparison  of  the  old  and  new 
bubble  forms,  one  generic  question  pertaining  to  the  length  of  time  the 
provider  had  been  filling  out  the  forms,  one  question  regarding  adoption  of 
the  form  Army-wide,  and  one  open-ended  question  designed  to  elicit  provider 
comments  on  the  study. 

Section  five  was  primarily  for  the  clinic  chiefs  and  consisted  of  five 
questions,  four  of  which  related  to  the  use  and  value  of  information  derived 

» 

from  the  bubble  forms.  The  final  question  was  a  theoretical  compliance 
question.  A  copy  of  the  Health  Care  Provider  Survey  is  located  at  Appendix  A. 

Administration  of  ihe  BcotiAer  Sm.ey 

During  the  reliability  phase  of  the  study  (Moon  et  al,  1989),  the  study 
team  provided  each  point  of  contact  (POC)  at  the  study  site  with  a  large 
number  of  provider  surveys  (between  100  and  150  surveys  depending  on  hospital 
size).  Each  of  the  POCs  was  then  instructed  to  provide  each  participating 
provider  with  a  copy  of  the  survey,  allowing  a  reasonable  time  for  completion. 

The  amount  of  time  given  to  each  provider  varied  by  hospital  and  mission 
requirements  (i.e.,  no  major  training  exercises  or  hospital  inspections). 

LlmilatioDj? 

The  (unavailability  of  providers  who  participated  in  the  study  was  a 
major  limitation  ir>  this  project.  This  was  the  result  of  rapid  staff  turnover 
caused  in  most  cases  by  training  or  teaching  requirements.  For  example, 

Brooke  Army  Medical  Center,  Ft.  Sam  Houston,  Texas,  is  a  teaching  hospital 
which  provides  medical  training  in  nearly  all  specialties.  Womack  Army 
Community  Hospital,  Ft.  Bragg,  North  Carolina,  and  Blanchfield  Army  Community 
Hospital,  Fort  Campbell,  Kentucky,  are  major  training  centers  for  the  Family 
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Practice  Specialty.  Additionally,  since  the  study  period  covered  nearly  two 
calendar  years,  and  the  normal  tour  of  duty  is  three  calendar  years,  each  year 
a  number  of  providers  were  reassigned  to  other  medical  treatment  facilities  or 
completed  their  service  obligation  and  left.  No  attempt  was  made  to  locate 
these  providers,  therefore  an  assumption  is  made  that  those  who  did  respond 
were  representative  of  the  whole  number  of  providers. 

RESULTS 

Section  1  -  Demographic  Mis 

Since  the  individual  POCs  used  different  approaches  to  the  administration 
and  collection  of  the  provider  surveys,  the  number  of  returned  questionnaires 
varied  by  hospital.  However,  there  was  an  order  to  this  variation  as  the 
largest  hospital  in  this  study,  Brooke  Army  Medical  Center,  Fort  Sam  Houston, 
Texas,  returned  the  most  provider  surveys;  the  smallest  hospital  in  the 
study,  Fox  Army  Community  Hospital,  returned  the  fewest  number.  A  total  of 
493  Provider  Surveys  was  returned.  (Appendix  B,  Table  B-l,  contains  the 
number  of  returned  surveys  for  each  hospital.) 

The  most  frequent  length  of  time  that  the  providers  had  either  served  in 
the  Army  and  worked  in  a  medical  treatment  facility  (MTF)  or  were  employed  by 
civil  service  in  a  medical  treatment  facility  (MTF)  was  2-6  years.  (See 
Appendix  B,  Table  B-2,  for  a  frequency  distribution  of  provider's  length  of 
service  time.) 

In  terms  of  the  rank  of  the  providers,  the  Officer  Corps  represented  over 
50  percent  of  the  health  care  providers.  (A  complete  listing  of  rank  and  pay 
grade  of  providers  is  found  in  Appendix  B,  Table  B-3.) 

As  one  might  expect,  the  variety  of  specialty  areas  of  the  providers  was 
quite  extensive.  However,  the  largest  group  of  specialty  providers  was  from 
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Family  Practice.  (Appendix  B,  Table  B-4  contains  a  listing  of  the  most 
frequent  provider  respondents  by  specialty). 

Results  &£  Section  JI  -  Old  Bubble  Form 

This  section  addressed  issues  related  to  the  use  of  the  "old"  bubble 
forms  (forms  used  prior  to  1  Hay  1987).  A  majority  of  providers,  nearly  41 
percent,  indicated  they  only  completed  Section  3  (Provider  I.D.  and  time  spent 
with  patient),  Section  4  (Evaluation,  Services  and  Procedures)  and  Section  5 
(Diagnosis/Reason  for  visit).  A  complete  description  of  the  sections  of  the 
old  bubble  forms  completed  by  the  providers  is  contained  in  Appendix  C,  Table 
C-l. 

About  33  percent  of  the  providers  surveyed  indicated  they  completed  more 
than  20  bubble  forms  a  day,  while  less  than  11  percent  indicated  that  they 
filled  out  fewer  than  five  bubble  forms  per  day.  A  minimal  average  of  12 
forms  per  day  can  be  calculated  by  giving  those  providers  who  indicated  that 
they  completed  fewer  than  five  forms  a  day  a  value  of  "one  form  a  day"  and 
those  providers  who  completed  more  than  20  forms  a  day  a  value  of  "21  forms  a 
day."  A  more  detailed  account  of  old  bubble  form  usage  is  contained  in 
Appendix  C,  Table  C-2. 

The  amount  of  time  required  to  complete  the  bubble  form  was  essentially 
equally  divided  among  three  groups  of  providers:  Those  requiring  21-40 
seconds,  those  needing  41-60  seconds,  and  those  using  more  than  60  seconds. 

A  categorical  summary  of  old  form  completion  times  is  located  in  Appendix  C, 
Table  C-3. 

Over  76  percent  of  the  providers  surveyed  indicated  they  almost  always 
completed  a  bubble  form  on  a  patient.  As  a  coronary,  only  two  percent 
indicated  they  almost  never  completed  a  bubble  form  on  the  patient.  Further 
information  on  the  frequency  of  provider  compliance  pertaining  to  the  old 
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bubble  forms  is  contained  in  Appendix  C,  Table  C-4. 

In  terms  of  accuracy,  more  than  85  percent  of  the  providers  indicated 
that  they  usually  or  almost  always  were  accurate,  while  only  one  percent 
reported  they  were  never  accurate.  For  further  information  on  accuracy  of 
provider  information  on  old  bubble  forms,  see  Appendix  C,  Table  C-5. 

Additional  information  in  this  area  was  obtained  from  the  question  asking 
providers  their  perceptions  of  the  accuracy  of  other  providers.  Sixty-five 
percent  of  the  providers  answered  that  their  colleagues  usually  or  almost 
always  were  accurate.  Less  than  one  percent  answered  that  they  thought  their 
colleagues  were  almost  never  accurate.  Appendix  C,  Table  C-6,  contains  a 
table  of  providers'  perception  of  how  accurately  other  providers  completed  the 
old  bubble  forms. 

The  final  four  questions  in  Section  II  addressed  logistical  issues  in  the 
design  of  the  old  bubble  form.  Approximately  68  percent  of  the  providers 
indicated  that  "more  than  75%  of  the  time"  they  could  find  the  desired 
evaluation/service/procedure  and  fewer  than  4  percent  of  the  providers 
answered  that  "less  than  25%  of  the  time"  could  they  find  the  appropriate 
evaluation/service/procedure.  A  more  detailed  analysis  regarding  the 
availability  of  evaluation/service/procedure  codes  on  the  old  bubble  forms  is 
found  in  Appendix  C,  Table  C-7. 

Only  32  percent  of  the  providers  indicated  they  were  "very  satisfied  or 
satisfied"  with  the  arrangement  of  the  evaluation/service/procedures  listed  on 
the  old  bubble  forms.  Nearly  52  percent  of  the  providers  were  either 
"somewhat  dissatisfied  or  dissatisfied"  with  the  arrangement  of  this  section. 

A  more  detailed  analysis  of  providers'  views  on  this  issue  is  contained  in 
Appendix  C,  Table  C-8. 

Nearly  65  percent  of  the  providers  noted  that  more  than  75  percent  of  the 
time  they  were  able  to  find  the  desired  "primary  reason  for  visit/diagnoses" 
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on  the  form.  Only  five  percent  of  providers  reported  that  less  than  25 
percent  of  the  time  could  they  find  the  appropriate  primary  reason  for  visit 
or  diagnoses  on  the  old  bubble  form.  Appendix  C,  Table  C-9,  contains  more 
information  on  this  question. 

In  terms  of  satisfaction,  38  percent  indicated  they  were  satisfied  with 
the  arrangement  of  the  primary  reason  for  visit/diagnoses  on  the  form. 

However,  43  percent  of  the  providers  were  either  somewhat  dissatisfied  or 
dissatisfied  with  the  arrangement  of  the  primary  reason  for  visit/diagnoses  on 
the  old  bubble  form.  (Appendix  C,  Table  C-10,  contains  a  complete  summary  of 
the  providers'  responses  to  this  question.) 

Rasul  te  fll  SaaLten  ILL  -  Hew  Babble  Form 

The  questions  contained  in  Section  III  pertained  to  the  new  bubble  forms 
(forms  used  after  May  1,  1987).  These  bubble  forms  went  through  a  substantial 
revision  with  the  primary  objective  being  to  make  the  forms  easier  for  the 
provider  to  use. 

Almost  49  percent  of  the  providers  indicated  they  completed  those 
sections  which  addressed  clinical  issues  (i.e.,  provider  identification 
number,  time  spent  with  patient,  evaluation/services/procedures  and  diagnosis/ 
reason  for  visit).  Nearly  31  percent  indicated  they  completed  the  entire 
new  bubble  form.  (See  Appendix  0,  Table  D-l,  for  more  information  on  this 
question.) 

The  use  of  the  new  bubble  forms  was  fairly  consistent  throughout  the 
categories  with  slightly  more  providers  (31  percent)  indicating  they  completed 
between  11  and  20  forms  per  day.  Moreover,  fewer  than  11  percent  of  the 
providers  completed  fewer  than  five  new  forms  a  day.  (Appendix  D,  Table  D-2, 
contains  more  information  on  use  of  the  new  forms.) 

The  time  necessary  to  complete  the  new  bubble  form  varied  from  less  than 
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20  seconds  to  more  than  a  minute.  The  majority  of  providers  (32  percent) 
indicated  they  required  21-40  seconds  to  complete  the  new  Hubble  form. 

However,  nearly  21  percent  indicated  they  required  more  than  a  minute  to  do 
so.  (See  Appendix  D,  Table  0-3  for  more  information  on  this  question.) 

More  than  90  percent  of  the  providers  surveyed  indicated  they  usually  or 
almost  always  completed  a  bubble  form  on  the  patients  they  treated. 

This  is  in  contrast  to  the  one  percent  who  reported  they  almost  never 
completed  a  new  bubble  form  on  a  patient.  (For  additional  information  on  the 
frequency  of  forms  completed  by  providers,  see  Appendix  D,  Table  D-4.) 

In  terms  of  the  accuracy  of  the  data  entered  on  the  new  bubble  forms, 
slightly  more  than  80  percent  of  the  providers  indicated  that  the  information 
they  entered  was  almost  always  or  usually  accurate.  Conversely,  slightly  more 
than  five  percent  of  the  providers  answered  that  the  information  they  entered 
on  the  new  bubble  forms  was  seldom  accurate  or  almost  never  accurate. 

(Appendix  D,  Table  D-5  contains  a  complete  analysis  of  the  accuracy  of  data 
entered  by  the  providers.)  Of  the  providers  surveyed,  68  percent  believed 
that  the  other  providers  in  their  clinic  were  almost  always  accurate  or 
usually  accurate  in  entering  data  in  the  bubble  forms.  Fewer  than  six  percent 
of  the  providers  indicated  that  the  other  providers  in  their  clinic  were 
seldom  accurate  or  never  accurate  in  entering  data  on  the  new  forms.  (For 
additional  information  on  this  topic,  see  Appendix  D,  Table  D-6.) 

Since  the  study  providers  were  requested  to  do  additional  work  (i.e. 
complete  the  bubble  forms),  the  study  team  was  interested  in  determining  the 
effect  of  filling  out  the  bubble  form  on  provider  workload.  The  effects 
proved  to  be  mixed,  with  about  33  percent  of  the  respondents  indicating  that 
completing  tLe  bubble  form  had  no  effect  on  their  workload  and  29  percent 
indicating  that  patients  waited  longer  for  care.  However,  only  about  three 
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percent  believed  they  had  seen  fewer  patients  as  a  result  of  completing  the 
new  bubble  forms.  (See  Appendix  D,  Table  D-7  for  additional  provider 
responses.) 

About  67  percent  of  the  providers  indicated  that  75  percent  of  the  time 
they  were  able  to  locate  the  evaluations/services/procedures  that  they 
performed.  Around  five  percent  of  the  providers  responded  that  less  than  25 
percent  of  the  time  were  they  able  to  locate  the  performed  evaluations, 
services  or  procedures  on  the  new  bubble  form.  (Appendix  D,  Table  D-8 
contains  more  detailed  provider  responses  to  this  issue.) 

In  terms  of  satisfaction  with  the  arrangement  of  the  evaluation/ 
services/procedures  section  of  the  new  bubble  form  approximately  44  percent  of 
the  providers  indicated  that  they  were  either  very  satisfied  or  satisfied 
with  the  arrangement  of  the  items  contained  in  this  section.  However,  nearly 
38  percent  of  the  providers  were  either  somewhat  dissatisfied  or  dissatisfied 
with  the  arrangement  of  this  section  on  the  new  bubble  forms.  (See  Appendix 
D,  Table  D-9  for  more  information  on  this  topic.) 

Nearly  63  percent  of  the  providers  indicated  that  in  75  percent  or  more 
of  the  patient  visits  they  were  able  to  locate  the  primary  reason  for  visit 
and,  if  appropriate,  secondary  diagnoses.  Only  about  five  percent  noted  they 
could  not  locate  the  appropriate  diagnosis  or  reason  for  visit  in  25  percent 
or  fewer  visits.  (Appendix  0,  Table  D-10  contains  a  complete  summary  of  this 
question  on  the  new  bubble  form.) 

In  terms  of  provider  satisfaction  with  the  arrangement  of  the  primary 
reason  for  visit  and  secondary  diagnoses,  44  percent  of  the  providers 
indicated  that  they  were  either  very  satisfied  or  satisfied  with  the 
arrangement.  However,  37  percent  were  either  somewhat  dissatisfied  or 
dissatisfied  with  the  arrangement  of  the  primary  reason  for  visit  and 
secondary  diagnoses  on  the  new  bubble  forms.  (A  full  description  of 
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providers'  responses  to  this  question  is  contained  in  Appendix  D,  Table  D-ll.) 

Results  af  Section  H  -  Mil  Old  and  Mfitt  Bubble  Form? 

Section  IV  was  designed  for  users  of  both  the  old  and  new  forms  to 
evaluate  the  usefulness  of  the  project.  Of  those  providers  who  used  both 
forms,  slightly  more  than  58  percent  believed  that  the  evaluations/services/ 
procedures  section  was  improved  while  14  percent  thought  the  section  was  not 

♦ 

as  good.  (See  Appendix  E,  Table  E-l.) 

‘  In  terms  of  the  primary  reason  for  visit  and  secondary  diagnoses  section, 

the  new  form  was  rated  as  an  improvement  over  the  old  form  by  nearly  55%  of 
the  providers.  Still,  the  old  form  had  its  supporters;  almost  17  percent  of 
the  providers  preferred  this  section  as  it  was  on  the  old  form.  (See  Appendix 
E,  Table  E-2,  for  additional  information.) 

Approximately  62  percent  of  the  providers  who  completed  the  survey 
had  been  filling  out  the  bubble  forms  for  more  than  12  months.  Only  around 
four  percent  indicated  they  had  been  completing  the  form  for  less  than  2 
months.  (See  Appendix  E,  Table  E-3,  for  a  complete  analysis  of  the  length  of 
time  the  providers  had  been  completing  the  forms.)  When  queried  as  to  whether 
or  not  the  bubble  forms  should  be  adopted  Army-wide,  74  percent  responded 
negatively,  13  percent  answered  affirmatively  with  the  remaining  13  percent 
v  being  undecided.  (See  Appendix  E,  Table  E-4.) 

The  final  item  in  this  section  was  an  open-ended  question  designed  to 
elicit  general  comments  regarding  the  project.  A  total  of  302  responses  was 
made  by  the  providers.  A  content  analysis  (Scott  &  Wertheimer,  1966)  of  these 
responses  resulted  in  the  creation  of  33  categories.  The  four  major  categories 
included  (1)  completing  the  bubble  forms  was  too  time  consuming  (38 
respondents),  (2)  the  bubble  form  was  an  additional  burden/too  time  consuming 
(37  respondents),  (3)  the  project  was  a  waste  of  time  and/or  money  (37 
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respondents),  and  (4)  completing  the  forms  detracted  from  patient  care.  A 
complete  description  of  the  categories  and  the  frequency  of  responses  is 
contained  in  Appendix  F. 

Re&ulls  flf  Section  1  -  Clinic  Chiefs'  Comneats 

The  final  section  of  the  ACDB  Provider  Survey  was  designed  primarily  for 
clinic  chiefs,  and  related  to  the  use  of  the  bubble  form  data.  Almost  41 
percent  of  the  clinic  chiefs  indicated  they  had  received  monthly  reports 
derived  from  data  contained  in  the  bubble  forms.  However,  almost  24  percent 
of  the  clinic  chiefs  noted  they  had  never  received  a  report  from  their  bubble 
form  data.  Additional  information  on  this  question  is  contained  in  Appendix 
6,  Table  G-l. 

In  terms  of  the  usefulness  of  the  information,  exactly  60  percent  of  the 
clinic  chiefs  who  received  the  information  found  it  useful  to  some  degree 
while  40  percent  indicated  that  it  was  not  useful.  Appendix  G,  Table  G-2, 
contains  more  detailed  information  on  this  question. 

Some  16  percent  of  the  clinic  chiefs  indicated  that  they  were  not 
interested  in  receiving  additional  reports  that  could  be  provided  from  the 
bubble  form  data.  However,  more  than  70  percent  of  the  clinic  chiefs 
expressed  some  degree  of  interest  in  receiving  additional  reports.  The 
remaining  14  percent  were  unsure  about  receiving  additional  reports.  (See 
Appendix  G,  Table  G-3,  for  more  information  on  this  question.) 

Slightly  more  than  45  percent  of  the  clinic  chiefs  who  responded  to  the 
survey  believed  that  the  clinical  information  collected  on  the  bubble  form 
would  be  useful  to  the  Army  Medical  Department.  The  remaining  55  percent  of 
the  respondents  were  essentially  divided  between  not  being  sure  of  the 
usefulness  of  the  data  to  the  Army  (27%)  and  the  data  not  being  useful  to  the 
Army  (28%).  (See  Appendix  G,  Table  G-4.) 


12 
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The  final  question  for  the  clinic  chiefs  was  a  theoretical  compliance 
question.  The  chiefs  were  asked  how  they  "would  go  about  gaining  compliance 
among  staff  providers  if  the  bubble  form  would  be  used  as  a  billing  form."  A 
total  of  115  responses  were  collected  from  the  clinic  chiefs.  By  employing  a 
content  analysis  methodology,  researchers  grouped  these  responses  into  26 
major  response  categories.  The  largest  category  which  accounted  for  nearly  16 
percent  of  the  responses  was  "the  need  to  hire  more  clerical  help  to  save 
provider  time."  The  second  largest  category,  representing  13  percent  of  the 
provider  responses,  included  comments  such  as  "inform/educate  staff  regarding 
importance  of  forms  for  getting  resources  and  staffing."  The  third  largest 
category,  representing  12  percent  of  the  provider  responses,  was  related  to 
recommendations  such  as  "track  individual  compliance/provide  monthly 
feedback/punish  noncompliance."  None  of  the  other  major  categories  of 
provider  responses  accounted  for  more  than  six  percent  of  the  total  number  of 
responses.  A  complete  summary  of  the  categories  and  the  number  of  responses 
for  each  of  the  categories  is  contained  in  Appendix  H. 

DISCUSSION 

The  participation  of  nearly  500  health  care  providers  in  completing  the 
ACDB  provider  survey  has  provided  the  Army  Medical  Department  and  ACDB  study 
team  with  many  valuable  insights  on  the  project.  Some  of  the  more  salient 
points  will  be  discussed  in  this  section. 

One  of  the  most  useful  findings  is  knowing  that  the  providers  completed 
bubble  forms  on  nearly  every  patient  for  whom  they  provided  care.  This 
finding  has  a  number  of  very  important  implications.  First  and  foremost,  it 
provides  the  study  team  with  an  additional  measure  of  confidence  in  the  fact 
that  the  collected  data  is  an  accurate  representation  of  the  existing  workload 
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in  the  ambulatory  clinics  of  the  hospitals  that  participated  in  the  study. 

Secondly,  when  this  finding  is  combined  with  the  finding  that  more  than 
85  percent  of  the  providers  indicated  that  they  are  usually  or  almost  always 
accurate  in  the  information  they  entered,  the  level  of  confidence  increases 
even  more.  This  makes  the  data  base  more  relevant  for  workload  estimation  as 
well  as  for  epidemiological  studies  of  incidence  of  illnesses  for  various 
groups.  Thirdly,  this  perceived  accuracy  of  data  by  providers  is  supported  by 
the  findings  of  the  ACDB  Reliability  Study  (Moon,  et  al.  1989)  which 
demonstrated  that  the  data  entered  on  the  bubble  forms  was  extremely  accurate 
and  was  as  good  as  any  data  within  or  outside  the  United  States  Army  Medical 
Department. 

The  finding  that  nearly  68  percent  of  the  providers  could  find  the 
appropriate  evaluations/services/procedures  75  percent  or  more  of  the  time 
indicates  that  the  types  of  procedures  performed  in  the  various  out-patient 
clinics  are  performed  with  a  good  deal  of  consistency.  Additionally,  should 
the  Army  Medical  Department  or  the  Department  of  Defense  proceed  with  plans  to 
develop  a  clinically  based  management  system  like  the  Composite  Health  Care 
System  (CHCS),  the  procedures  lists  developed  for  the  ACDB  study  could  serve 
as  the  basis  from  which  to  develop  a  more  accurate  procedures  list.  This  is 
also  true  for  the  development  of  a  specialty  related  menu  of  diagnoses. 

Another  finding  which  merits  comment  is  the  "effect  that  completing  the 
bubble  form  had  on  provider  workloads."  The  initial  effects  appeared  mixed 
with  about  33  percent  of  the  providers  indicating  that  completing  the  bubble 
forms  had  no  effect  on  their  workload  and  29  percent  responding  that  patients 
waited  longer  for  care.  This  was  in  reality  not  as  significant  as  one  might 
initially  believe. 

An  additional  analysis  proved  enlightening.  This  analysis  consisted  of 
taking  the  average  number  of  forms  completed  in  a  day  (14.9)  and  multiplying 
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it  by  the  average  time  required  to  complete  a  form  (42.3  seconds).  The  result 
is  ten  and  one  half  minutes  per  day  per  provider.  Therefore,  for  those 
providers  (29%)  indicating  that  patients  waited  longer  for  care  or  for  the 
providers  who  worked  longer  hours,  the  amount  of  time  the  patients  were 
waiting  or  providers  were  working  must  have  been  minimal.  Moreover, 
additional  analyses  indicated  that  as  a  provider  became  more  familiar  with  a 
form,  his  proficiency  increased  and  the  time  required  to  complete  the  form 
decreased.  Thus  the  additional  workload  which  resulted  from  using  the 
bubble  forms  decreased  over  time.  (See  Appendix  H.) 

CONCLUSIONS 

The  first  objective  of  the  study  was  to  solicit  input  from  the  health 
care  providers/participants  regarding  the  adequacy  of  the  individual  specialty 
clinical  menus.  This  was  accomplished  through  the  review  of  nearly  500 
completed  provider  surveys  representing  a  broad  range  of  specialties. 

The  second  study  objective  was  to  obtain  a  self-report  measure  from 
the  providers'  perspectives  on  the  accuracy  of  data  entered  in  the  ACDB 
clinical  bubble  forms.  This  was  achieved  through  a  number  of  questions 
contained  in  the  provider  survey,  and  the  findings  were  validated  by  a 
previous  study  (Moon  et  al ,  1989). 

The  third  objective  of  the  study  was  to  acquire  a  measure  of  the  amount 
of  time  necessary  for  providers  to  complete  the  bubble  forms.  The  findings 
revealed  a  decrease  in  time  required  for  completion  of  the  new  bubble 
form  versus  the  old  bubble  form.  Average  time  for  completion  of  the  old 
bubble  form  was  49.4  seconds  compared  with  42.3  seconds  for  the  new  bubble 
form.  This  is  a  decrease  of  slightly  over  7  seconds  of  time  required  for 
completing  the  new  form.  An  analysis  of  the  data  also  indicated  that  the  more 
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forms  completed  by  a  provider  the  less  time  required  to  complete  each  form. 

The  fourth  and  final  study  objective  was  to  provide  the  health  care 
providers  participating  in  the  study  the  opportunity  to  express  their  views  on 
the  study.  This  objective  was  met  by  obtaining  over  300  written  comments  by 
the  providers  on  the  project. 

The  participation  of  nearly  500  health  care  providers  in  completing  the 
Health  Care  Provider  Survey  has  afforded  the  Army  Medical  Department  with 
valuable  insights  on  many  aspects  of  the  ACDB  study.  # 

SUMMARY 

The  Army's  ACDB  study  collected  information  on  more  than  3.1  million 
patient  visits  during  a  21-month  period;  more  than  4,000  providers  were 
involved  at  six  medical  treatment  facilities.  During  the  data  collection 
period,  various  study  team  members  collected  anecdotal  information  from  a 
number  of  participating  providers.  Unfortunately,  this  information  was  never 
consolidated  and  validated.  To  overcome  this  deficiency  and  to  give  study 
participants  (the  health  care  providers)  with  the  opportunity  to  provide  input 
or  to  evaluate  the  ACDB  study,  the  ACDB  Provider  Survey  was  developed. 

The  provider  survey  collected  information  from  nearly  500  health  care 
providers  and  has  provided  the  Army  Medical  Department  with  valuable  insight 
on  many  aspects  of  the  ACDB  Study.  However,  the  most  important  aspect  may  be 
in  the  knowledge  that  the  data  contained  in  the  bubble  forms  not  only  are  % 

valid  but  also  are  an  accurate  representation  of  the  care  provided  in  the 

* 

outpatient  clinics  as  measured  ;by  the  responses  of  the  participating 
providers. 
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APPENDIX  A 


* 


DEPARTMENT  OF  THE  ARMY 
HEAOQUARTERS,  UNITED  STATES  ARMY  HEALTH  SERVICES  COMMAND 
FORT  SAM  HOUSTON.  TEXAS  78234-6000 

REPLY  TO 
ATTENTION  OP: 

10  August  1987 

MEMORANDUM  FOR:  Ambulatory  Care  Data  Base  Health  Care  Providers 
SUBJECT:  Health  Care  Provider  Survey 

1.  The  data  collection  phase  of  the  Ambulatory  Care  Data  Base  (ACDB)  project 
is  ending.  After  all  remaining  “bubble  forms"  are  scanned,  the  central  data 
base  will  contain  almost  3  million  encounter  records  representing  ambulatory 
medical  practice  in  more  than  50  different  specialties  from  the  6  medical 
treatment  facilities  which  served  as  test  sites.  These  data  and  the  analysis 
being  planned  are  extremely  important  to  the  future  of  the  Army  Medical 
Department  (AMEDO).  Your  efforts  in  completing  the  mark  sense  encounter  forms 
are  greatly  appreciated  by  both  The  Surgeon  General  and  me. 

2.  The  study  is  now  in  the  evaluation  phase,  an  important  part  of  which  is 
assessing  provider  experience  and  satisfaction  with  the  bubble  forms.  To 
assist  the  investigators  at  the  U.S.  Army  Health  Care  Studies  and  Clinical 
Investigation  Activity  (HCSCIA)  with  this  evaluation,  please  take  a  few 
minutes  to  answer  the  attached  questions  and  return  the  questionnaire  to  your 
local  point  of  contact.  The  questionnaire  will  take  you  less  than  10  minutes 
to  complete. 

3.  Your  participation  and  that  of  your  fellow  health  care  providers  is 
crucial  to  the  completion  of  the  ACDB  study.  Future  efforts  in  designing  data 
capture  methods  and  in  developing  and  utilizing  appropriate  “menus"  of 
diagnoses  and  procedures  to  encompass  the  range  of  practice  in  each  specialty 
and  for  each  provider  type  will  be  based,  in  part,  on  your  response. 

4.  Thank  you  very  much  for  your  thoughtful  participation  and  assistance. 


0  , 

TRACY  E.  STREVEY,  JR.' 

Major  General ,  MC 

Command i no 


HSHN-? 
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AMBULATORY  CARE  DATA  BASE  PROVIDER  SURVEY 


SECTION  I 

(Please  circle  the  appropriate  response.) 


A.  How  long  have  you  been  in  the  Army  and  working  in  a  Medical  Treatment 
Facility  (MTF)  or  employed  by  Civil  Service  at  a  MTF? 

1.  Less  than  2  years 

2.  2  to  6  years 

3.  7  to  10  years 

4.  Over  10  years 


B.  What  is  your  present  rank/pay  grade? 

1.  E-l  to  E-5 

2.  E-6  to  E-9 

3.  Officer  Warrant 

4.  Officer  0-1  to  0-3 

5.  Officer  0-4  to  0-6 

6.  Civilian  GS  1  to  GS  7 

7.  Civilian  GS  8  to  GS  16 

8.  Personal  Service  Contract  Civilian 


c. 

In  which  specialty 

area  do  you  work  most  of  the  time? 

01. 

Adolescent 

18.  General  Surgery 

36. 

Pain  Control 

02. 

Allergy 

19.  Gynecology 

37. 

Pediatrics 

03. 

Audiology 

20.  Immunizations 

38. 

Physical  Medicine 

04. 

Cardiology 

21.  Infectious  Disease 

39. 

Physical  Therapy 

05. 

Cardiothorac  Surg 

22.  Inhalation/Resp  Ther 

40. 

Plastic  Surgery 

06. 

Cast 

23.  Internal  Medicine 

41. 

Podiatry 

07. 

Comm  Health  Nurse 

24.  Nephrology/Dialysis 

42. 

Preventive  Med 

08. 

Brace/Ortho  Appl 

25.  Neurology 

43. 

Primary  Care 

09. 

Dermatology 

26.  Neurosurgery 

(AMIC/ACC/Med  Exam) 

10. 

EFMP 

27.  Nutrition 

44. 

Psychiatry 

11. 

EKG 

28.  Obstetrics 

45. 

Psychology 

12. 

Emergency  Room 

29.  Occupational  Health 

46. 

Pulmonary 

13. 

Endocrine 

30.  Occupational  Therapy 

47. 

Rheumatology 

14. 

ENT 

31.  Oncology /Hematology 

48. 

Social  Work 

15. 

Family  Practice 

32.  Ophthalmology 

49. 

Speech  Pathology 

16. 

Flight  Medicine 

33.  Optometry 

50. 

Troop  Med  Clinic 

17. 

Gastroenterology 

34.  Orthopedics 

51. 

Urology 

35.  Otorhinolaryngology 

52. 

Other 

DO  NOT  USE 
THIS  SPACE 

ID 

(1-6) 

_ (7) 

_ (8) 


(9,10) 
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SECTION  II 


Section  II  pertains  to  BU8BLE  FORMS  used  before  1  May  1987  which  will  be 
referred  to  as  OLD  BUBBLE  FORMS.  If  you  filled  out  the  OLD  BUBBLE  FORMS, 
please  answer  the  questions  in  this  section,  otherwise  skip  to  Section  III, 


A.  Indicate  ALL  the  portions  of  the  OLD  BUBBLE  FORM  you  NORMALLY  completed 
(you  may  choose  more  than  one). 

1.  PATIENT  SECTION  -  Date,  Sponsor's  Soc  Sec  No  &  Family  Member  Prefix 

2.  ADMINISTRATION  SECTION  -  UCA  Data,  Place  of  Visit,  Appt  Status,  Status 

of  Visit 

3.  PROVIDER  SECTION  -  Provider  ID,  Time  Spent  with  Patient 

4.  EVALUATIONS/SERVICES/PROCEDURES 

5.  DIAGNOSIS/REASON  FOR  VISIT 


B.  Approximately  how  many  OLD  BUBBLE  FORMS  did  you  fill  out  on  an  average 
clinic  day? 


1.  Less  than  5  a  day 

2.  6-10  a  day 

3.  11-20  a  day 

4.  More  than  20  a  day 

5.  Not  sure 


C.  For  the  portions  of  the  OLD  BUBBLE  FORM  that  you  NORMALLY  completed,  how 
much  time  on  the  average  did  you  spend  on  each  form? 

1.  Less  than  20  seconds 

2.  21-40  seconds 

3.  41-60  seconds 

4.  Over  60  seconds 


D.  How  often  did  you  fill  out  an  OLD  BUBBLE  FORM  on  your  patients? 

1.  Almost  always 

2.  Usually 

3.  About  half  the  time 

4.  Seldom 

5.  Almost  never 

6.  Not  sure 


E.  How  accurate  is  the  information  that  you  marked  on  the  OLD  BUBBLE  FORM? 


(19) 


1.  Almost  always  accurate 

2.  Usually  accurate 

3.  Accurate  about  half  the  time 

4.  Seldom  accurate 

5.  Almost  never  accurate 

F.  In  your  opinion  how  accurate  is  the  information  that  OTHER  PROVIDERS  in 
your  CLINIC  marked  on  the  OLD  BUBBLE  FORM? 

1.  Almost  always  accurate 

2.  Usually  accurate 

3.  Accurate  about  half  the  time 

4.  Seldom  accurate 

5.  Almost  never  accurate 

6.  Not  Sure 

G.  Approximately  what  percentaae  of  the  time  were  you  able  to  find  (on 
the  OLD  BUBBLE  FORM)  the  EVALUATIONS/SERVICES/PROCEDURES  that  you 
perform  in  the  outpatient  setting? 

1.  Around  90  percent 

2.  Around  75  percent 

3.  Around  50  percent 

4.  Around  25  percent 

5.  Less  than  25  percent 

H.  How  satisfied  are  you  with  the  arrangement  of  the  EVALUATIONS/SERVICES/ 
PROCEDURES  on  the  OLD  BUBBLE  FORM? 

1.  Very  satisfied 

2.  Satisfied 

3.  Not  Sure 

4.  Somewhat  dissatisfied 

5.  Dissatisfied 

I.  Approximately  what  percentage  of  the  time  were  you  able  to  find  the 
PRIMARY  REASON  FOR  VISIT  and  SECONDARIES  (DIAGNOSES)  on  the  OLD  BUBBLE 
FORM? 

1.  Around  90  percent 

2.  Around  75  percent 

3.  Around  50  percent 

4.  Around  25  percent 

5.  Less  than  25  percent 

J.  How  satisfied  were  you  with  the  arrangement  of  the  PRIMARY  REASON  FOR 
VISIT  AND  SECONDARIES  on  the  OLD  BUBBLE  FORM? 

1.  Very  satisfied 

2.  Satisfied 

3.  Not  Sure 

4.  Somewhat  dissatisfied 

5.  Dissatisfied 


(20) 


_ (21) 


(22) 


_ (23) 


(24) 
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SECTION  III 


SECTION  III  pertains  to  the  BUBBLE  FORMS  used  after  1  May  1987  which  will  be 
referred  to  as  the  NEW  BUBBLE  FORMS.  Please  complete  this  section  is  you  used 
the  NEW  BUBBLE  FORMS. 


A.  Mark  all  the  portions  of  the  NEW  BUBBLE  FORM  you  normally  completed  (you 
may  choose  more  than  one  section). 

1.  ADMINISTRATIVE  SECTION  -  Date,  Sponsor's  Soc  Sec  No  and  Family  Member 

Prefix,  &  optional  fills  for  UCA,  Appt  status,  etc. 

2.  PROVIDER  SECTION  -  Provider  ID,  Time  Spent  with  Patient  &  optional  fill 

for  Job  Rel  Ill/Inj,  Military  Disposition,  etc. 

3.  EVALUATIONS/SERVICES/PROCEDURES 

4.  DIAGNOSIS/REASON  FOR  VISIT 


B.  Approximately  how  many  NEW  BUBBLE  FORMS  did  you  fill  out  on  an  average 
clinic  day? 


1.  Less  than  5  a  day 

2.  6-10  a  day 

3.  11-20  a  day 

4.  More  than  20  a  day 

5.  Not  sure 


C.  For  the  portions  of  the  NEW  BUBBLE  FORM  that  you  NORMALLY  completed,  how 
much  time  on  the  average  did  you  spend  on  each  form? 

1.  Less  than  20  seconds 

2.  21  to  40  seconds 

3.  41  to  60  seconds 

4.  Over  60  seconds 


D.  How  often  did  you  fill  out  a  NEW  BUBBLE  FORM  on  your  patients? 

1.  Almost  always 

2.  Usually 

3.  About  half  the  time 

4.  Seldom 

5.  Almost  never 

6.  Not  sure 


E.  How  accurate  is  the  information  that  you  marked  on  the  NEW  BUBBLE  FORM? 

1.  Almost  always  accurate 

2.  Usually  accurate 

3.  Accurate  about  half  the  time 

4.  Seldom  accurate 

5.  Almost  never  accurate 


F.  In  vour  opinion  how  accurate  is  the  information  that  OTHER  PROVIDERS  in 
your  CLINIC  marked  on  the  NEW  BUBBLE  FORM? 

1.  Almost  always  accurate 

2.  Usually  accurate 

3.  Accurate  about  half  the  time 

4.  Seldom  accurate 

5.  Almost  never  accurate 

6.  Not  Sure 

6.  What  was  the  effect  filling  out  the  NEW  BUBBLE  FORM  had  on  your  workload? 
(Choose  more  than  one,  if  applicable.) 

1.  No  effect 

2.  Patients  waited  longer  for  care 

3.  I  saw  fewer  patients 

4.  I  worked  longer  hours 

H.  Approximately  what  percentage  of  the  time  were  you  able  to  find  (on  the 
NEW  BUBBLE  FORM)  the  EVALUATIONS/SERVICES/PROCEDURES  that  you  perform 
in  the  outpatient  setting? 

1.  Around  90  percent 

2.  Around  75  percent 

3.  Around  50  percent 

4.  Around  25  percent 

5.  Less  than  25  percent 

I.  How  satisfied  are  you  with  the  arrangement  of  the  EVALUATIONS/SERVICES/ 
PROCEDURES  on  the  NEW  BUBBLE  FORM? 

1.  Very  satisfied 

2.  Satisfied 

3.  Not  Sure 

-  4.  Somewhat  dissatisfied 

5.  Dissatisfied 

J.  Approximately  what  percentage  of  the  time  were  you  able  to  find  the 
PRIMARY  REASON  FOR  VISIT  and  SECONDARIES  (DIAGNOSES)  on  the  NEW  BUBBLE 
FORM? 

1.  Around  90  percent 

2.  Around  75  percent 

3.  Around  50  percent 

4.  Around  25  percent 

5.  Less  than  25  percent 

K.  How  satisfied  are  you  with  the  arrangement  of  the  PRIMARY  REASON  FOR  VISIT 
AND  SECONDARIES  on  the  NEW  BUBBLE  FORM? 

1.  Very  satisfied 

2.  Satisfied 

3.  Not  Sure 

4.  Somewhat  dissatisfied 

5.  Dissatisfied 


(34) 


_ (35) 
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SECTION  IV 


Users  of  BOTH  the  OLD  and  NEW  BU88LE  FORMS  should  complete  this  section. 

A.  In  your  PROFESSIONAL  OPINION  how  would  you  rate  the  EVALUATIONS/SERVICES/  _ (39) 

PROCEDURES  section  of  the  NEW  BUBBLE  FORM  compared  to  the  same  section  of 

the  OLD  BUBBLE  FORM? 

1.  Did  not  use  old  form 

2.  Greatly  improved 

3.  Moderately  improved 

4.  Improved 

5.  About  the  same 

6.  Not  as  good 

B.  In  your  PROFESSIONAL  OPINION  how  would  you  rate  the  PRIMARY  REASON  FOR  _ (40) 

VISIT  AND  SECONDARIES  (DIAGNOSIS)  section  of  the  NEW  BUBBLE  FORM 

compared  to  the  PRIMARY  REASON  FOR  VISIT  AND  SECONDARIES  (DIAGNOSIS) 
section  of  the  OLD  BUBBLE  FORM? 

1.  Did  not  use  old  form 

2.  Greatly  improved 

3.  Moderately  improved 

4.  Improved 

5.  About  the  same 

6.  Not  as  good 

C.  How  long  have  you  been  filling  out  the  BUBBLE  FORMS?  _ (41) 

1.  Less  than  2  months 

2.  2  to  4  months 

3.  5  to  9  months 

4.  9  to  12  months 

5.  Over  12  months 

D.  Do  you  believe  the  encounter  form  such  as  the  one  you  have  been  using  _ (42) 

should  be  adopted  Army-wide? 

1.  Yes 

2.  No 

3.  Don't  know 

E.  Thank  you  for  completing  this  survey.  Is  there  anything  else  you  want  to  _ (43) 

add? 

COMMENTS: 
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(CLINIC  CHIEFS,  Please  complete  back  side.) 


AMBULATORY  CARE  DATA  BASE  PROVIDER  SURVEY 
ADDITIONAL  QUESTIONS  FOR  CLINIC  CHIEFS 


A.  How  often  do  you  receive  reports  or  information  from  the  BUBBLE  Forms?  _ (44) 

1.  More  than  once  a  month 

2.  Once  a  month 

3.  Every  other  month 

A.  Less  than  every  other  month 
5.  Never 

B.  How  useful  is  the  information  (for  you  or  vour  clinic)  that  you  receive  (45) 

from  the  BUBBLE  FORMS? 

1.  Very  useful 

2.  Useful 

3.  Moderately  useful 
A.  Marginally  useful 

5.  Not  useful 

6.  Never  receive  any  information 

C.  If  additional  information  on  your  patients  could  be  provided  to  you  from  _ (46) 

the  BUBBLE  forms,  how  interested  would  you  be  in  receiving  it? 

1.  Very  interested 

2.  Somewhat  interested 

3.  Not  sure 

4.  Possibly  interested 

5.  Not  at  all  interested 

D.  In  your  PROFESSIONAL  OPINION  do  you  think  the  clinical  information  _ (47) 

collected  on  the  BUBBLE  FORM  will  be  useful  to  the  Army  Medical  Department? 

1.  Very  useful 

2.  Moderately  useful 

3.  Useful 

4.  Not  sure 

5.  Not  useful 

5.  Assuming  that  the  BUBBLE  FORM  would  be  used  as  a  "BILLING  FORM"  for  _ (48) 

workload  documentation  and  justifying  resources,  i.e.,  staff,  equipment, 
training,  etc.,  how  would  YOU  go  about  gaining  COMPLIANCE  among  providers? 
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APPENDIX  B 
TABLE  B-l 


NUMBER  OF  RETURNED  PROVIDER  SURVEYS 
BY 

STUDY  SITE 
(N=493) 


STUDY  SITE 

NUMBER  OF  RETURNED  SURVEYS 

Brooke  Army  Medical  Center 
(Fort  Sam  Houston) 

Texas 

188 

Blanchfield  Army  Community  Hospital 
(Fort  Campbell) 

Kentucky 

103 

Womack  Army  Community  Hospital 
(Fort  Bragg) 

North  Carolina 

94 

Bayne- Jones  Army  Community  Hospital 
(Fort  Polk) 

Louisiana 

62 

Moncrief  Army  Community  Hospital 
(Fort  Jackson) 

South  Carolina 

29 

Fox  Army  Community  Hospital 
(Redstone  Arsenal) 

Alabama 

17 

TOTAL 

493 
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APPENDIX  B 
TABLE  B-2 


DEMOGRAPHIC  INFORMATION 
PROVIDER  LENGTH  OF  SERVICE 
(N=493) 


PROVIDER  LENGTH  OF  SERVICE 

NUMBER 

Less  than  2  years 

80 

2-6  years 

182 

7-10  years 

98 

Over  10  years 

133 

TOTAL 

493 

APPENDIX  B 
TABLE  B-3 

DEMOGRAPHIC  INFORMATION 
PROVIDER  RANK  OR  PAY  GRADE 
(N=  492) 


RANK  OR  PAY  GRADE 

NUMBER 

E-l  to  E-5 

81 

E-6  to  E-9 

30 

Warrant  Officer 

13 

Officer  0-1  to  0-3 

119 

Officer  0-4  to  0-6 

133 

Civilian  GS-1  to  GS-7 

52 

Civilian  GS-8  to  GS-16 

61 

Personal  Service  Contract  (Civilian) 

3 

TOTAL 

492 
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APPENDIX  B 
TABLE  B-4 

MOST  FREQUENT  PROVIDER  RESPONDENTS 
BY  SPECIALTY* 

N  >  10 


SPECIALTY  NUMBER 


Family  Practice  35 
Emergency  Room  34 
Internal  Medicine  33 
Pediatrics  31 
Troop  Medical  Clinic  28 
Physical  Therapy  25 
Gynecology  18 
Occupational  Health  17 
Ophthalmology  16 
Social  Work  16 
Occupational  Therapy  14 
Oncology/Hematology  14 
Allergy  13 
Orthopedics  13 
Podiatry  11 


TOTAL  318 


*  There  was  a  total  of  51  specialties  listed  by  the  providers. 
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APPENDIX  C 
TABLE  C-l 


SECTIONS  OF  OLD  BUBBLE  FORMS 

COMPLETED  BY  PROVIDERS 
N=424 


SECTION  COMPLETED  NUMBER  OF  PROVIDERS 


Didn't  complete  any  sections 

37 

Eval uati on/servi ces/procedures 
section  and  diagnosis/reason  for 
visit  section 

23 

Provider  section,  evaluation/services/ 
procedures  section  and  diagnosis/ 
reason  for  visit  section 

172 

Patient  section,  provider  section,  evaluation/ 
services/procedures  section  &  diagnosis/ 
reason  for  visit  section 

20 

Administration  section,  provider  section, 
evaluations/services/procedures  section  & 
diagnosis/reason  for  visit  section 

37 

All  the  sections  on  the  form 

135 

TOTAL 

424 
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APPENDIX  C 
TABLE  C-2 


PROVIDER  USAGE  OF  OLD  BUBBLE  FORM 
N=465 

NUMBER  OF  FORMS  COMPLETED  PER  DAY  NUMBER  OF  PROVIDERS 


Less  than  5 

50 

Between  6  and  10 

117 

Between  11  and  20 

134 

More  than  20 

153 

Unsure 

11 

TOTAL 

465 

APPENDIX  C 

TABLE  C-3 

PROVIDER  TIME  REQUIRED 

TO 

COMPLETE  THE  OLD  BUBBLE  FORM 

N=466 

TIME  IN  SECONDS  NUMBER  OF  PROVIDERS 

Less  than  20  seconds 

33 

Between  21  and  40  seconds 

138 

Between  41  and  60  seconds 

148 

More  than  60  seconds 

147 

TOTAL 

466 
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APPENDIX  C 
TABLE  C-4 


FREQUENCY  OF  OLD  BUBBLE  FORMS 
COMPLETED  ON  PATIENTS 
N=468 

FREQUENCY  NUMBER  OF  PROVIDERS 


Almost  always 

360 

Usually 

62 

About  half  the  time 

28 

Seldom 

6 

Almost  never 

10 

Not  sure 

2 

TOTAL 

468 

APPENDIX  C 

TABLE  C-5 

ACCURACY  OF  PROVIDER  INFORMATION 

ON  OLD  BUBBLE  FORMS 

N=467 

DEGREE  OF  ACCURACY  NUMBER  OF  PROVIDERS 

Almost  always  accurate 

185 

Usually  accurate 

213 

Accurate  about  half  the  time 

50 

Seldom  accurate 

13 

Almost  never  accurate 

6 

Not  sure 

0 

TOTAL 

467 
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APPENDIX  C 
TABLE  C-6 

PROVIDER  PERCEPTIONS  ON  HOW  ACCURATELY  OTHER  PROVIDERS 
COMPLETED  THE  OLD  BUBBLE  FORM 
Ns465 

DEGREE  OF  ACCURACY  NUMBER  OF  PROVIDERS 

Almost  always  accurate 
Usually  accurate 
Accurate  about  half  the  time 
Seldom  accurate 
Almost  never  accurate 
Not  sure 

TOTAL  465 

APPENDIX  C 
TABLE  C-7 

PERCENTAGE  OF  TIME  PROVIDERS  COULD  LOCATE 
THE  APPROPRIATE  EVALUATIONS/SERVICES/PROCEDURES 
ON  THE  OLD  BUBBLE  FORMS 
N=463 

PERCENTAGE  OF  TIME  ITEM  LOCATED  NUMBER  OF  PROVIDERS 

About  90  percent  of  the  time 
About  75  percent  of  the  time 
About  50  percent  of  the  time 
About  25  percent  of  the  time 
Less  than  25  percent  of  the  time 

TOTAL  463 


129 

184 

113 

20 

17 


112 

192 

96 

18 

4 

43 
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APPENDIX  C 
TABLE  C-8 


PROVIDER  SATISFACTION  WITH  THE  ARRANGEMENT 
ON  THE  OLD  BUBBLE  FORM  OF 
THE  EVALUATIONS/SERVICES/PROCEDURES  SECTION 

N=463 

DEGREE  OF  PROVIDER  SATISFACTION  NUMBER  OF  PROVIDERS 


Very  satisfied 

14 

* 

Satisfied 

136 

* 

Not  sure 

73 

Somewhat  dissatisfied 

117 

Dissatisfied 

123 

TOTAL  463 


APPENDIX  C 
TABLE  C-9 

PERCENTAGE  OF  TIME  PROVIDER  WAS  ABLE  TO  LOCATE 
REASON  FOR  VISIT/DIAGNOSES 
ON  THE  OLD  BUBBLE  FORMS 
N=462 


PERCENTAGE  OF  TIME  ITEM  LOCATED  NUMBER  OF  PROVIDERS 


About  90  percent  of  the  time 

106 

About  75  percent  of  the  time 

194 

About  50  percent  of  the  time 

117 

About  25  percent  of  the  time 

21 

Less  than  25  percent  of  the  time 

24 

TOTAL 

462 

45 


APPENDIX  C 


TABLE  C-10 

PROVIDER  SATISFACTION  WITH  THE  ARRANGEMENT 
OF  THE  OLD  BUBBLE  FORM 

PRIMARY  REASON  FOR  VISIT/DIAGNOSES  SECTION 
N=464 


DEGREE  OF  PROVIDER  SATISFACTION  NUMBER  OF  PROVIDERS 


Very  satisfied 

18 

Satisfied 

160 

Not  sure 

84 

Somewhat  dissatisfied 

114 

Dissatisfied 

88 

TOTAL 

464 

46 


APPENDIX  D 
TABLE  D-l 


SECTIONS  OF  NEW  BUBBLE  FORMS 
COMPLETED  BY  PROVIDERS 
N=411 


NAME  OF  SECTION  NUMBER  OF  PROVIDERS 


Didn't  complete  any  sections 

41 

Evaluations/services/procedures 
section  and  diagnosis/reason  for 
visit  section 

43 

Provider  section,  evaluations/services/ 
procedures  section  and  diagnosis/ 
and  reason  for  visit  section 

200 

Patient  section/provider  section  / 
evaluations/services/procedures  section 
and  diagnosis/reason  for  visit  section 

127 

TOTAL 

411 

APPENDIX  D 

TABLE  D-2 

PROVIDER  USAGE  OF  NEW  BUBBLE 
N=468 

FORM 

NUMBER  OF  FORMS  COMPLETED  PER  DAY 

NUMBER  OF  PROVIDERS 

Less  than  5 

50 

Between  6  and  10 

122 

Between  11  and  20 

146 

More  than  20 

138 

Unsure 

12 

TOTAL 


468 


APPENDIX  D 
TABLE  D-3 

PROVIDER  TIME  REQUIRED 
TO 

COMPLETE  THE  NEW  BUBBLE  FORM 
N=467 


TIME  IN  SECONDS 

NUMBER  OF  PROVIDERS 

Less  than  20  seconds 

79 

Between  21  and  40  seconds 

150 

Between  41  and  60  seconds 

142 

More  than  60  seconds 

96 

TOTAL 

467 

APPENDIX  D 
TABLE  D-4 


FREQUENCY  OF  TIME  WHEN  NEW  BUBBLE 
WERE  COMPLETED  ON  PATIENTS 
N=468 

FORMS 

FREQUENCY 

NUMBER  OF  PROVIDERS 

Almost  always 

341 

Usually 

81 

About  half  the  time 

28 

Seldom 

11 

Almost  never 

5 

Not  sure 

2 

TOTAL 


468 


APPENDIX  D 
TABLE  D-5 


ACCURACY  OF  PROVIDER  INFORMATION 
ON  NEW  BUBBLE  FORMS 
N=468 

DEGREE  OF  ACCURACY  NUMBER  OF  PROVIDERS 


Almost  always  accurate 

219 

Usually  accurate 

157 

Accurate  about  half  the  time 

67 

Seldom  accurate 

16 

Almost  never  accurate 

9 

Not  sure 

0 

TOTAL 

468 

APPENDIX  D 

TABLE  D-6 

PROVIDER  PERCEPTION  OF  HOW  ACCURATELY  OTHER  PROVIDERS 
COMPLETED  THE  NEW  BUBBLE  FORM 

N=466 

DEGREE  OF  ACCURACY  NUMBER  OF  PROVIDERS 

Almost  always  accurate 

127 

Usually  accurate 

190 

Accurate  about  half  the  time 

71 

Seldom  accurate 

22 

Almost  never  accurate 

6 

Not  sure 

50 

TOTAL 

466 

51 


APPENDIX  D 

TABLE  D-7 

EFFECTS  ON  PATIENT  CARE 
AS  A  RESULT  OF  COMPLETING 
THE  NEW  BUBBLE  FORMS 
N=450 

EFFECT 

PROVIDER  RESPONSE 

No  effect 

150 

Patients  waited  longer  for  care 

131 

I  saw  fewer  patients 

14 

I  worked  longer  hours 

88 

Patients  waited  longer  for  care  and 

I  worked  longer  hours 

53 

Patients  waited  longer  for  care,  I  saw 
fewer  patients  and  I  worked  longer  hours 

14 

TOTAL 

450 

APPENDIX  D 
TABLE  D-8 

PERCENTAGE  OF  TIME  PROVIDERS  COULD  LOCATE 
THE  APPROPRIATE  EVALUATIONS/SERVICES/PROCEDURES 
ON  THE  NEW  BUBBLE  FORMS 


N=464 

PERCENTAGE  OF  TIME  ITEM  LOCATED 

NUMBER  OF  PROVIDERS 

Around  90  percent  of  the  time 

139 

Around  75  percent  of  the  time 

170 

Around  50  percent  of  the  time 

106 

Around  25  percent  of  the  time 

28 

Less  than  25  percent  of  the  time 

21 

TOTAL 

464 

52 


APPENDIX  D 

TABLE  D-9 

PROVIDER  SATISFACTION  WITH  THE  ARRANGEMENT 

OF  THE  EVALUATIONS/SERVICES/PROCEDURES  SECTION 

ON  THE  NEW  BUBBLE  FORM 

N=467 

DEGREE  OF  PROVIDER  SATISFACTION 

NUMBER  OF  PROVIDERS 

Very  satisfied 

38 

» 

Satisfied 

168 

Not  sure 

84 

p 

Somewhat  Dissatisfied 

100 

Dissatisfied 

77 

TOTAL 

467 

APPENDIX  D 

TABLE  D-10 

PERCENTAGE  OF  TIME 

PROVIDER  WAS  ABLE  TO  LOCATE  REASON  FOR  VISIT/DIAGNOSES 

ON  NEW  BUBBLE  FORM 

N=464 

■» 

PERCENTAGE  OF  TIME  ITEM  LOCATED 

NUMBER  OF  PROVIDERS 

% 

About  90  percent  of  the  time 

131 

About  75  percent  of  the  time 

159 

About  50  percent  of  the  time 

114 

About  25  percent  of  the  time 

37 

Less  than  25  percent  of  the  time 

23 

TOTAL 

464 

53 

1 

APPENDIX  D 


TABLE  D-ll 

PROVIDER  SATISFACTION  WITH  THE  ARRANGEMENT  OF 

THE  EVALUATIONS/SERVICES/PROCEDURES  SECTION 

OF  THE  NEW  BUBBLE  FORM 
N=465 


DEGREE  OF  SATISFACTION 


NUMBER  OF  PROVIDERS 


Very  satisfied 

35 

Satisfied 

169 

Not  sure 

88 

Somewhat  dissatisfied 

98 

Dissatisfied 

75 

TOTAL 

465 

54 


i 


APPENDIX  E 


APPENDIX  E 
TABLE  E-l 

PROVIDER  COMPARISON  OF  THE 
EVALUATIONS/SERVICES/PROCEDURES  SECTION  OF 
THE  OLD  AND  NEW  BUBBLE  FORMS 
N=452 


PROVIDER  RESPONSE 

NUMBER  OF  PROVIDERS 

Did  not  use  old  form 

13 

Greatly  improved 

57 

Moderately  improved 

94 

Improved 

105 

About  the  same 

121 

Not  as  good 

62 

TOTAL 

452 

APPENDIX  E 
TABLE  E-2 

PROVIDER  COMPARISON  OF  THE 
PRIMARY  REASON  FOR  VISIT/DIAGNOSIS  SECTION  OF  THE 
OLD  AND  NEW  BUBBLE  FORMS 
N=453 


PROVIDER  RESPONSE 

NUMBER  OF  PROVIDERS 

Did  not  use  old  form 

12 

Greatly  improved 

53 

Moderately  improved 

90 

Improved 

99 

About  the  same 

125 

Not  as  good 

74 

TOTAL 

453 

57 


APPENDIX  E 
TABLE  E-3 


LENGTH  OF  TIME  PROVIDERS  WERE  COMPLETING 
BOTH  OLD  AND  NEW  BUBBLE  FORMS 
N=457 


LENGTH  OF  TIME 

NUMBER  OF  PROVIDERS 

Less  than  2  months 

19 

Between  2  and  4  months 

18 

Between  5  and  9  months 

58 

Between  9  and  12  months 

79 

Over  12  months 

283 

TOTAL 

457 

APPENDIX  E 
TABLE  E-4 

QUESTION  FOR  USERS  OF  BOTH  OLD  AND  NEW  FORMS 
REGARDING  ARMY-WIDE  ADOPTION  OF  THE  FORMS 
N=463 


RESPONSE 


NUMBER  OF  PROVIDERS 


Yes 

69 

No 

336 

Don't  know 

58 

TOTAL 


463 


58 


APPENDIX  F 

PROVIDERS'  GENERAL  COMMENTS  REGARDING  THE  ACDB  PROJECT 

N=302 

COMMENT  NUMBER  OF  PROVIDERS 


1. 

Too  time  consuming. 

38 

2. 

Additional  burden/too  much  work. 

37 

3. 

Waste  of  time  and/or  money. 

37 

4. 

Forms  detracted  from  patient  care. 

23 

5. 

Needed  additional  clerical  help  for  completion 
of  data  on  forms. 

18 

6. 

Feel  data  collected  not  accurate  ( i . e . ,  used 
approximate  Diagnosis,  etc. 

15 

7. 

Poor  design/bad  plan 

12 

8. 

Didn't  like  revised  Diagnoses  &  Procedures 
lists  (i.e.,  not  enough  Diagnoses,  etc.) 

12 

9. 

Should  discontinue  use. 

12 

10. 

Didn't  like. 

11 

11. 

Forms  need  further  improvement/simplification 

10 

12. 

Duplication  of  effort 

10 

13. 

Good  means  of  showing  accountability 

10 

14. 

Needed  feedback. 

9 

15. 

Not  useful  to  providers  or  patients,  an 
administrative  tool  only 

8 

16. 

Good  idea,  but . 

7 

17. 

Felt  'not  heard'  regarding  input  for 
forms  revision. 

5 

18. 

Bar  Code  System  should  be  used. 

3 

19. 

Forms  were  improved  in  organization. 

3 

20. 

Feedback  received  not  accurate 

3 

61 


APPENDIX  F 

PROVIDERS'  GENERAL  COMMENTS  REGARDING  THE  ACDB  PROJECT  (Continued) 


COMMENT 


NUMBER  OF  PROVIDERS 


21.  Highly  recommend  continued  use.  3 

22.  Felt  deceived  regarding  stated  purpose  of  forms/ 

length  of  study.  2 

23.  Used  reports  from  data  collected  to  track  injury 

trends/clinic  activities,  etc.  2 

24.  Recommend  Army-wide  usage  if  other  systems  of 

counting  workload  are  abandoned.  2 

25.  Switch  to  a  computerized  system  cross-referenced 

with  ICD-9-CM  Codes  in  a  mainframe  2 

26.  Saw  possible  worldwide  database  through  interface 

through  w/MIS  in  the  medical  treatment  facilities.  1 

27.  "Time  spent  with  patient"  is  a  useless  statistic  1 

28.  Should  keep  Disposition  Box  1 

29.  Hope  gains  are  worth  the  time  spent  1 

30.  Should  retain  patient  designations  of  "old"  and 

"new"  as  this  explains  time  spent  1 

31.  Money  is  needed  more  for  other  more  important 

areas  of  patient  care,  i.e.,  TDY  for  CMEs,  support 
personnel,  etc.  1 

32.  Fire  the  person  who  inflicted  bubble  forms  on  us.  1 

33.  The  busier  the  department,  the  more  likely  data 

will  be  inaccurate  and  underestimate  workload.  1 
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APPENDIX  6 
TABLE  G-l 

CLINIC  CHIEFS'  RESPONSES  REGARDING 
FREQUENCY  OF  BUBBLE  FORM  REPORTS 
N=123 


RESPONSE 

NUMBER  OF  PROVIDERS 

More  than  once  a  month 

4 

Once  a  month 

50 

Every  other  month 

6 

Less  than  every  other  month 

34 

Never 

29 

TOTAL 

123 

APPENDIX  G 
TABLE  G-2 

CLINIC  CHIEFS'  RESPONSES  REGARDING 
USEFULNESS  OF  BUBBLE  FORM  REPORTS 
N=126 


RESPONSES 

NUMBER  OF  PROVIDERS 

Very  useful 

6 

Useful 

11 

Moderately  useful 

14 

Marginally  useful 

29 

Not  useful 

40 

Never  received  information 

26 

TOTAL 

126 

65 


APPENDIX  G 
TABLE  G-3 


CLINIC  CHIEFS'  RESPONSES  REGARDING 
DEGREE  OF  INTEREST  IN  RECEIVING  ADDITIONAL 
PATIENT  INFORMATION 
N=124 


RESPONSES 

NUMBER  OF  PROVIDERS 

Very  interested 

24 

Somewhat  interested 

36 

Not  sure 

17 

Possibly  interested 

27 

Not  at  all  interested 

20 

TOTAL 

124 

APPENDIX  G 
TABLE  G-4 

CLINIC  CHIEFS'  RESPONSES  REGARDING 
USEFULNESS  OF  CLINICAL  INFORMATION 
N=126 


RESPONSES  NUMBER  OF  PROVIDERS 


Very  useful 

20 

Moderately  useful 

20 

Useful 

17 

Not  sure 

34 

Not  useful 

35 

TOTAL 

126 

66 


APPENDIX  H 

CLINIC  CHIEFS'  GENERAL  COMMENTS  REGARDING 
INCREASING  PROVIDER  COMPLIANCE 
N=115 

COMMENT  NUMBER  OF  PROVIDERS 


1.  Hire  more  clerical  help  to  save  providers'  time.  18 

2.  Inform/educate  staff  regarding  importance  of  forms 

for  getting  resources  and  staffing.  15 

3.  Track  individual  compliance/provide  monthly 

«  feedback/punish  noncompliance.  14 

4.  Simplify/shorten  form.  7 

5.  Don't  think  bubble  sheets  can  be  used  to  justify 

workload.  7 

6.  Eliminate  duplicate  reports.  5 

7.  Incentive  awards  (i.e.f  personal  recognition, 

compensatory  time,  money).  4 

8.  Make  it  mandatory.  4 

9.  Can't  gain  compliance  without  drastic  measures 

(i.e.,  pay/VCMJ)  4 

10.  Would  not  gain  compliance  because  system  was  not 

working  for  capturing  data.  4 

11.  Don't  know.  4 

12.  Crosscheck  bubble  sheets  against  sign-in  sheets  4 

.  13.  Directly  relate  resources  to  bubble  sheet  data  3 

14.  Use  hand/other  type  computer  for  tabulating.  3 

15.  Monitc^  statistics  gathered  from  forms  and  feedback 

to  providers  useful  information  on  clinic  workload, 
types  of  patient  visits,  etc.  3 

16.  Provide  quarterly  feedback  from  Commander,  HSC.  2 

17.  Would  have  to  see  fewer  patients  or  find  some  other 

way  to  gather  data.  2 

18.  Menus  need  to  be  more  precise  to  capture  accurate 

picture.  2 
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19.  Make  turning  in  the  correct  number  of  forms  the 

provider's  ticket  out  of  each  clinic  session.  2 

20.  No  problem.  2 

21.  Give  classes  in  how  to  complete  forms  properly.  1 

22.  Allow  48-hr  backload  per  provider  1 

23.  Break  leg/arm.  Put  a  hit  contract  on  rebels.  1 

24.  Use  as  4th  copy  for  SF558.  For  each  SF558,  must 

have  one  "billing  form."  1 

25.  Because  of  inaccuracy  and  noncompliance,  it  is  a 

waste  of  time.  1 

26.  More  resources  required  in  order  to  provide  adequate 

services.  1 
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